Multitemporal data analysis

ABSTRACT

A system for multitemporal data analysis is provided, comprising a directed computation graph service module configured to receive input data from a plurality of sources, analyze the input data to determine a best course of action for analyzing the input data, and split the input data for queueing to a general transformer service module or a decomposable service module based at least in part by analysis of the input data; a general transformer service module configured to receive data from the directed computation graph service module, and perform analysis on the received data; and a general transformer service module configured to receive data from directed computational graph module, and perform analysis on the received data.

CROSS-REFERENCE TO RELATED APPLICATIONS

Priority is claimed in the application data sheet to the followingpatents or patent applications, the entire written description of eachof which is expressly incorporated herein by reference in its entirety:

-   -   Ser. No. 15/790,206    -   62/569,362    -   Ser. No. 15/616,427    -   Ser. No. 14/925,974

BACKGROUND OF THE INVENTION Field of the Invention

The disclosure relates to the field of machine learning, particularly togeneral and decomposable data analysis.

Discussion of the State of the Art

Data analysis may often be required to be done on massive amounts ofdata. Even though the data may be labeled in one form or another, thedata may not have a uniform format since it may originate from differentsources, or the data may contain a lot of irrelevant data which may needto be formalized for maximum analysis efficiency. The data may alsocontain elements that may be better suited for other means of analysisthat is not provided by the current system. Creating a data analysismodel from scratch may be daunting, and manually curating such largeamounts of data may prove to be a tedious and time-consuming task.

Another trend that is growing in popularity is the concept of aserverless application, in which a developer does not have to create abackend server infrastructure for their application. The developer mayuser a Platform as a Service (PaaS) solution such as AMAZON LAMBDA tosimplify their backend requirements. Unlike analyzing large amounts ofdata, a serverless application may require a system with real-timestreaming data-handling capabilities. The computer systems used maydiffer as well, since a system well-suited for analyzing large amountsof data may be not able to analyze real-time streaming data.

Therefore, what is needed is a system that can programmatically analyzeboth large amounts of stored data, or streams of real-time data. Such asystem should allow a user to easily create, share, and distribute dataanalysis models. Such a system should also be flexible, and able to beused in many applications.

SUMMARY OF THE INVENTION

Accordingly, the inventor has conceived, and reduced to practice, asystem and method for multitemporal data analysis.

In a typical embodiment, a system is provided in which data may be inputinto a system by a user. The system determines the best course foranalyzing the data, which may include, without limitation, mapping andreducing the data, splitting general and decomposable data, and directedcomputation graph analysis. The data that may be put into the system mayinclude, without limitation, large and small amounts of stored data,live streaming data, and the like.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawings illustrate several aspects and, together withthe description, serve to explain the principles of the inventionaccording to the aspects. It will be appreciated by one skilled in theart that the particular arrangements illustrated in the drawings aremerely exemplary, and are not to be considered as limiting of the scopeof the invention or the claims herein in any way.

FIG. 1 is a diagram of an exemplary architecture of a business operatingsystem according to an embodiment of the invention.

FIG. 2 is a sequence flow diagram summarizing a method for taking datainput from a data source to perform analysis and functions with atransformer service as used in various embodiments of the invention.

FIG. 3 is a flowchart illustrating a method for data input and splittingfor multitemporal data analysis used in various embodiments of theinvention.

FIG. 4 is a flowchart illustrating a method for analyzing data using ageneral transformer service module as used in various embodiments of theinvention.

FIG. 5 is a flowchart illustrating a method for analyzing decomposabledata with a decomposable transformer service module as used in variousembodiments of the invention.

FIG. 6 is a block diagram illustrating an exemplary hardwarearchitecture of a computing device used in various embodiments of theinvention.

FIG. 7 is a block diagram illustrating an exemplary logical architecturefor a client device, according to various embodiments of the invention.

FIG. 8 is a block diagram illustrating an exemplary architecturalarrangement of clients, servers, and external services, according tovarious embodiments of the invention.

FIG. 9 is another block diagram illustrating an exemplary hardwarearchitecture of a computing device used in various embodiments of theinvention.

DETAILED DESCRIPTION

The inventor has conceived, and reduced to practice, a system and methodfor multitemporal data analysis.

One or more different aspects may be described in the presentapplication. Further, for one or more of the aspects described herein,numerous alternative arrangements may be described; it should beappreciated that these are presented for illustrative purposes only andare not limiting of the aspects contained herein or the claims presentedherein in any way. One or more of the arrangements may be widelyapplicable to numerous aspects, as may be readily apparent from thedisclosure. In general, arrangements are described in sufficient detailto enable those skilled in the art to practice one or more of theaspects, and it should be appreciated that other arrangements may beutilized and that structural, logical, software, electrical and otherchanges may be made without departing from the scope of the particularaspects. Particular features of one or more of the aspects describedherein may be described with reference to one or more particular aspectsor figures that form a part of the present disclosure, and in which areshown, by way of illustration, specific arrangements of one or more ofthe aspects. It should be appreciated, however, that such features arenot limited to usage in the one or more particular aspects or figureswith reference to which they are described. The present disclosure isneither a literal description of all arrangements of one or more of theaspects nor a listing of features of one or more of the aspects thatmust be present in all arrangements.

Headings of sections provided in this patent application and the titleof this patent application are for convenience only, and are not to betaken as limiting the disclosure in any way.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or morecommunication means or intermediaries, logical or physical.

A description of an aspect with several components in communication witheach other does not imply that all such components are required. To thecontrary, a variety of optional components may be described toillustrate a wide variety of possible aspects and in order to more fullyillustrate one or more aspects. Similarly, although process steps,method steps, algorithms or the like may be described in a sequentialorder, such processes, methods and algorithms may generally beconfigured to work in alternate orders, unless specifically stated tothe contrary. In other words, any sequence or order of steps that may bedescribed in this patent application does not, in and of itself,indicate a requirement that the steps be performed in that order. Thesteps of described processes may be performed in any order practical.Further, some steps may be performed simultaneously despite beingdescribed or implied as occurring non-simultaneously (e.g., because onestep is described after the other step). Moreover, the illustration of aprocess by its depiction in a drawing does not imply that theillustrated process is exclusive of other variations and modificationsthereto, does not imply that the illustrated process or any of its stepsare necessary to one or more of the aspects, and does not imply that theillustrated process is preferred. Also, steps are generally describedonce per aspect, but this does not mean they must occur once, or thatthey may only occur once each time a process, method, or algorithm iscarried out or executed. Some steps may be omitted in some aspects orsome occurrences, or some steps may be executed more than once in agiven aspect or occurrence.

When a single device or article is described herein, it will be readilyapparent that more than one device or article may be used in place of asingle device or article. Similarly, where more than one device orarticle is described herein, it will be readily apparent that a singledevice or article may be used in place of the more than one device orarticle.

The functionality or the features of a device may be alternativelyembodied by one or more other devices that are not explicitly describedas having such functionality or features. Thus, other aspects need notinclude the device itself.

Techniques and mechanisms described or referenced herein will sometimesbe described in singular form for clarity. However, it should beappreciated that particular aspects may include multiple iterations of atechnique or multiple instantiations of a mechanism unless notedotherwise. Process descriptions or blocks in figures should beunderstood as representing modules, segments, or portions of code whichinclude one or more executable instructions for implementing specificlogical functions or steps in the process. Alternate implementations areincluded within the scope of various aspects in which, for example,functions may be executed out of order from that shown or discussed,including substantially concurrently or in reverse order, depending onthe functionality involved, as would be understood by those havingordinary skill in the art.

Conceptual Architecture

FIG. 1 is a diagram of an exemplary architecture of a business operatingsystem 100 according to an embodiment of the invention. Client access tosystem 105 for specific data entry, system control and for interactionwith system output such as automated predictive decision making andplanning and alternate pathway simulations, occurs through the system'sdistributed, extensible high bandwidth cloud interface 110 which uses aversatile, robust web application driven interface for both input anddisplay of client-facing information and a data store 112 such as, butnot limited to MONGODB™, COUCHDB™, CASSANDRA™ or REDIS™ depending on theembodiment.

Much of the business data analyzed by the system both from sourceswithin the confines of the client business, and from cloud based sources107, public or proprietary such as, but not limited to: subscribedbusiness field specific data services, external remote sensors,subscribed satellite image and data feeds and web sites of interest tobusiness operations both general and field specific, also enter thesystem through the cloud interface 110, data being passed to theconnector module 135 which may possess the API routines 135 a needed toaccept and convert the external data and then pass the normalizedinformation to other analysis and transformation components of thesystem, the directed computational graph module 155, high volume webcrawler module 115, multidimensional time series database 120 and agraph stack service 145. Directed computational graph module 155retrieves one or more streams of data from a plurality of sources, whichincludes, but is not limited to, a plurality of physical sensors,network service providers, web based questionnaires and surveys,monitoring of electronic infrastructure, crowd sourcing campaigns, andhuman input device information. Within directed computational graphmodule 155, data may be split into two identical streams in aspecialized pre-programmed data pipeline 155 a, wherein one sub-streammay be sent for batch processing and storage while the other sub-streammay be reformatted for transformation pipeline analysis. The data may bethen transferred to a general transformer service module 160 for lineardata transformation as part of analysis or the decomposable transformerservice module 150 for branching or iterative transformations that arepart of analysis. Directed computational graph module 155 represents alldata as directed graphs where the transformations are nodes and theresult messages between transformations edges of the graph. High-volumeweb crawling module 115 may use multiple server hosted preprogrammed webspiders which, while autonomously configured, may be deployed within aweb scraping framework 115 a of which SCRAPY™ is an example, to identifyand retrieve data of interest from web based sources that are not welltagged by conventional web crawling technology. Multiple dimension timeseries data store module 120 may receive streaming data from a largeplurality of sensors that may be of several different types. Multipledimension time series data store module 120 may also store any timeseries data encountered by system 100 such as, but not limited to,environmental factors at insured client infrastructure sites, componentsensor readings and system logs of some or all insured client equipment,weather and catastrophic event reports for regions an insured clientoccupies, political communiques and/or news from regions hosting insuredclient infrastructure and network service information captures (such as,but not limited to, news, capital funding opportunities and financialfeeds, and sales, market condition), and service related customer data.Multiple dimension time series data store module 120 may accommodateirregular and high-volume surges by dynamically allotting networkbandwidth and server processing channels to process the incoming data.Inclusion of programming wrappers 120 a for languages—examples of whichmay include, but are not limited to, C++, PERL, PYTHON, andERLANG™—allows sophisticated programming logic to be added to defaultfunctions of multidimensional time series database 120 without intimateknowledge of the core programming, greatly extending breadth offunction. Data retrieved by multidimensional time series database 120and high-volume web crawling module 115 may be further analyzed andtransformed into task-optimized results by directed computational graph155 and associated general transformer service 160 and decomposabletransformer service 150 modules. Alternately, data from themultidimensional time series database and high-volume web crawlingmodules may be sent, often with scripted cuing information determiningimportant vertices 145 a, to graph stack service module 145 which,employing standardized protocols for converting streams of informationinto graph representations of that data, for example open graph internettechnology (although the invention is not reliant on any one standard).Through the steps, graph stack service module 145 represents data ingraphical form influenced by any pre-determined scripted modifications145 a and stores it in a graph-based data store 145 b such as GIRAPH™ ora key-value pair type data store REDIS™, or RIAK™, among others, any ofwhich are suitable for storing graph-based information.

Results of the transformative analysis process may then be combined withfurther client directives, additional business rules and practicesrelevant to the analysis and situational information external to thedata already available in automated planning service module 130, whichalso runs powerful information theory-based predictive statisticsfunctions and machine learning algorithms 130 a to allow future trendsand outcomes to be rapidly forecast based upon the current systemderived results and choosing each a plurality of possible businessdecisions. Then, using all or most available data, automated planningservice module 130 may propose business decisions most likely to resultin favorable business outcomes with a usably high level of certainty.Closely related to the automated planning service module 130 in the useof system-derived results in conjunction with possible externallysupplied additional information in the assistance of end user businessdecision making, action outcome simulation module 125 with a discreteevent simulator programming module 125 a coupled with an end user-facingobservation and state estimation service 140, which is highly scriptable140 b as circumstances require and has a game engine 140 a to morerealistically stage possible outcomes of business decisions underconsideration, allows business decision makers to investigate theprobable outcomes of choosing one pending course of action over anotherbased upon analysis of the current available data.

FIG. 2 is a sequence flow diagram summarizing a method 200 for takingdata input from a data source to perform analysis and functions with atransformer service as used in various embodiments of the invention. Atan initial step 205, data is input into a system configured to usebusiness operating system 100. The data may be, for example,pre-gathered data, or it may be data that is being gathered in real-timeduring analysis. The data is queued to a graph stack service module tobe converted into directed computational graph (DCG) form. Otherexamples of data may include, without limitation, data gathered bybusiness operating system 100 and stored in local or cloud data stores;data gathered, and aggregated in real-time via web crawling; largeamounts of; user-generated events caused by their actions in anapplication or website; and the like. At step 210, the data, now in DCGform, is queued to a DCG service module for graphical analysis. Analysismay include the system determining which transformer service the datashould be queued to for a best outcome for analysis. At this point,there may be two possible execution paths as indicated by marked box201. In the first execution path, the DCG data is determined by the DCGservice module to be appropriate for general transformer service 160 atstep 215. Some examples of data suitable for the general transformerservice may include, without limitation, large batches data, data storedon distributed databases such as RIAK, data that is generally suited forlinear operations, data gathered and stored from sensors or monitoringsoftware overtime, or the like.

In some cases, there may be an additional step 220. At step 220, duringdata analysis, there may be decomposable data elements within thegeneral data that may be extracted by business operating system 100 andqueued to decomposable transformer service for further analysis.

In the alternate execution path, the input data may be determined tocontain data suitable for decomposable transformer service module 150 atstep 225. The data is queued directly to the decomposable transformerservice module. Some examples of data suitable for the decomposabletransformer service module may include, without limitation, livestreaming data received from sensors or monitoring software, eventscaused from user action on a website or app, non-linear operations, newsocial media postings, and, without a loss of generality, highlyparallelizable tasks that don't share state. Besides decomposable dataanalysis, the real-time data handling capabilities of the decomposabletransformer service may be utilized as a maintenance-free backend thatmay be used for applications, and web development. This may enable adeveloper to focus on creating their software, and not have to worrybuilding a suitable backend infrastructure and maintaining it.

It will be appreciated by one skilled in the art that the dynamic dataanalyzing capabilities of this system allows for a multitude ofapplications for any amount of data. For instance, using the correctmodel for a particular query the system can handle the data gathering,parsing, and analysis. For example, a data analyst may want to get asense of what the general public thinks of certain political candidate.The analyst may develop his own model, download a model from arepository, or purchase a model created by another user to use in hissystem configured to run business operating system 100. The analyst mayconfigure his system to automate data gathering from social media feeds,news feeds, message board postings, and the like. The analyst's system,using the transformer services described herein along with otherfunctions of business operating system 100, may integrate the feeds, mapand summarize the data, analyze the sentiment from the gathered datausing the model, and generate a report based on the results.

Detailed Description of Exemplary Aspects

FIG. 3 is a flowchart illustrating a method 300 for data input andsplitting for multitemporal data analysis used in various embodiments ofthe invention. At an initial step 305, data is input into a systemconfigured to run business operating system 100. As mentioned above, thedata may comprise, for instance, user input, previously gathered data,data that is being gathered on-the-fly in real-time, or the like. Thedata may also be a combination of the multiple types previouslymentioned. At step 310, the input data is queued, and filtered by thesystem to collect the relevant parts of the data. At step 315, using DCGanalysis, the system may split the data, and determine the type of dataand the most appropriate module for further analysis depending on degreeof shared state information as part of a declarative formalism formessage passing between atomic workers (computing instances) in the poolof distributed computing resources. If the data is determined to beappropriate for the general transformer service module, step 320 isreached, wherein the process continues at step 405 in method 400, whichis discussed below. On the other hand, if the data is determined to beappropriate for the decomposable transformer service module, step 325 isreached, wherein the process may continue at step 505 in method 500,which is also discussed below.

FIG. 4 is a flowchart illustrating a method 400 for analyzing data usinga general transformer service module as used in various embodiments ofthe invention. At an initial step 405, general data is queued. Onesource of data is discussed in method 300. At step 410, the data isformalized into an efficient, database-friendly format and stored forprocessing. Storage may be handled by a distributed database solutionsuch as RIAK. At step 415, the data is broken up and mapped to a metricspecified by a user, and the mapped data is summarized based at least inpart by the specified metric. At step 420, further leveraging the DCGservice module for analysis, biases in the data may be determined. Anydecomposable elements in the data at split off and queued to thedecomposable transformer service module to step 425, a method in whichis discussed below in FIG. 5. While step 425 is occurring, the generaldata is aggregated and compiled into a report at step 430. At step 435,the system may perform an action pre-configured by a user. Actions mayinclude, for instance, a program function, sending an alert, activatinga trigger, or the like.

FIG. 5 is a flowchart illustrating a method 500 for analyzingdecomposable data with a decomposable transformer service module as usedin various embodiments of the invention. At an initial step 505,decomposable data is queued at the decomposable transformer servicemodule. At step 510, the system determines whether the operation shouldremain in an iterative loop. The loop may terminate, for example, whenthere is no more data to analyze, when a trigger is activated, when analert is received, or a pre-specified event has occurred. If no moreiterations are required, the cycle terminates, and an action isperformed at step 515. As mentioned above, actions may include, forinstance, a program function, sending an alert, activating a trigger, orthe like.

On the other hand, if the iterative cycle is still required, the systemdetermines whether the model used for analysis should be retrained withthe iterative data at step 520. If the system is determined to bestable, and the model does not need to be retrained, the system doesanother check to see whether it should remain in the iterative cycle.Otherwise, if the model is determined to require retraining, theiterated data is used to retrain the analysis model and redeployed atstep 525, before doing another iterative cycle check.

Hardware Architecture

Generally, the techniques disclosed herein may be implemented onhardware or a combination of software and hardware. For example, theymay be implemented in an operating system kernel, in a separate userprocess, in a library package bound into network applications, on aspecially constructed machine, on an application-specific integratedcircuit (ASIC), or on a network interface card.

Software/hardware hybrid implementations of at least some of the aspectsdisclosed herein may be implemented on a programmable network-residentmachine (which should be understood to include intermittently connectednetwork-aware machines) selectively activated or reconfigured by acomputer program stored in memory. Such network devices may havemultiple network interfaces that may be configured or designed toutilize different types of network communication protocols. A generalarchitecture for some of these machines may be described herein in orderto illustrate one or more exemplary means by which a given unit offunctionality may be implemented. According to specific aspects, atleast some of the features or functionalities of the various aspectsdisclosed herein may be implemented on one or more general-purposecomputers associated with one or more networks, such as for example anend-user computer system, a client computer, a network server or otherserver system, a mobile computing device (e.g., tablet computing device,mobile phone, smartphone, laptop, or other appropriate computingdevice), a consumer electronic device, a music player, or any othersuitable electronic device, router, switch, or other suitable device, orany combination thereof. In at least some aspects, at least some of thefeatures or functionalities of the various aspects disclosed herein maybe implemented in one or more virtualized computing environments (e.g.,network computing clouds, virtual machines hosted on one or morephysical computing machines, or other appropriate virtual environments).

Referring now to FIG. 6, there is shown a block diagram depicting anexemplary computing device 10 suitable for implementing at least aportion of the features or functionalities disclosed herein. Computingdevice 10 may be, for example, any one of the computing machines listedin the previous paragraph, or indeed any other electronic device capableof executing software- or hardware-based instructions according to oneor more programs stored in memory. Computing device 10 may be configuredto communicate with a plurality of other computing devices, such asclients or servers, over communications networks such as a wide areanetwork a metropolitan area network, a local area network, a wirelessnetwork, the Internet, or any other network, using known protocols forsuch communication, whether wireless or wired.

In one aspect, computing device 10 includes one or more centralprocessing units (CPU) 12, one or more interfaces 15, and one or morebusses 14 (such as a peripheral component interconnect (PCI) bus). Whenacting under the control of appropriate software or firmware, CPU 12 maybe responsible for implementing specific functions associated with thefunctions of a specifically configured computing device or machine. Forexample, in at least one aspect, a computing device 10 may be configuredor designed to function as a server system utilizing CPU 12, localmemory 11 and/or remote memory 16, and interface(s) 15. In at least oneaspect, CPU 12 may be caused to perform one or more of the differenttypes of functions and/or operations under the control of softwaremodules or components, which for example, may include an operatingsystem and any appropriate applications software, drivers, and the like.

CPU 12 may include one or more processors 13 such as, for example, aprocessor from one of the Intel, ARM, Qualcomm, and AMD families ofmicroprocessors. In some aspects, processors 13 may include speciallydesigned hardware such as application-specific integrated circuits(ASICs), electrically erasable programmable read-only memories(EEPROMs), field-programmable gate arrays (FPGAs), and so forth, forcontrolling operations of computing device 10. In a particular aspect, alocal memory 11 (such as non-volatile random access memory (RAM) and/orread-only memory (ROM), including for example one or more levels ofcached memory) may also form part of CPU 12. However, there are manydifferent ways in which memory may be coupled to system 10. Memory 11may be used for a variety of purposes such as, for example, cachingand/or storing data, programming instructions, and the like. It shouldbe further appreciated that CPU 12 may be one of a variety ofsystem-on-a-chip (SOC) type hardware that may include additionalhardware such as memory or graphics processing chips, such as a QUALCOMMSNAPDRAGON™ or SAMSUNG EXYNOS™ CPU as are becoming increasingly commonin the art, such as for use in mobile devices or integrated devices.

As used herein, the term “processor” is not limited merely to thoseintegrated circuits referred to in the art as a processor, a mobileprocessor, or a microprocessor, but broadly refers to a microcontroller,a microcomputer, a programmable logic controller, anapplication-specific integrated circuit, and any other programmablecircuit.

In one aspect, interfaces 15 are provided as network interface cards(NICs). Generally, NICs control the sending and receiving of datapackets over a computer network; other types of interfaces 15 may forexample support other peripherals used with computing device 10. Amongthe interfaces that may be provided are Ethernet interfaces, frame relayinterfaces, cable interfaces, DSL interfaces, token ring interfaces,graphics interfaces, and the like. In addition, various types ofinterfaces may be provided such as, for example, universal serial bus(USB), Serial, Ethernet, FIREWIRE™, THUNDERBOLT™, PCI, parallel, radiofrequency (RF), BLUETOOTH™, near-field communications (e.g., usingnear-field magnetics), 802.11 (WiFi), frame relay, TCP/IP, ISDN, fastEthernet interfaces, Gigabit Ethernet interfaces, Serial ATA (SATA) orexternal SATA (ESATA) interfaces, high-definition multimedia interface(HDMI), digital visual interface (DVI), analog or digital audiointerfaces, asynchronous transfer mode (ATM) interfaces, high-speedserial interface (HSSI) interfaces, Point of Sale (POS) interfaces,fiber data distributed interfaces (FDDIs), and the like. Generally, suchinterfaces 15 may include physical ports appropriate for communicationwith appropriate media. In some cases, they may also include anindependent processor (such as a dedicated audio or video processor, asis common in the art for high-fidelity AN hardware interfaces) and, insome instances, volatile and/or non-volatile memory (e.g., RAM).

Although the system shown in FIG. 6 illustrates one specificarchitecture for a computing device 10 for implementing one or more ofthe aspects described herein, it is by no means the only devicearchitecture on which at least a portion of the features and techniquesdescribed herein may be implemented. For example, architectures havingone or any number of processors 13 may be used, and such processors 13may be present in a single device or distributed among any number ofdevices. In one aspect, a single processor 13 handles communications aswell as routing computations, while in other aspects a separatededicated communications processor may be provided. In various aspects,different types of features or functionalities may be implemented in asystem according to the aspect that includes a client device (such as atablet device or smartphone running client software) and server systems(such as a server system described in more detail below).

Regardless of network device configuration, the system of an aspect mayemploy one or more memories or memory modules (such as, for example,remote memory block 16 and local memory 11) configured to store data,program instructions for the general-purpose network operations, orother information relating to the functionality of the aspects describedherein (or any combinations of the above). Program instructions maycontrol execution of or comprise an operating system and/or one or moreapplications, for example. Memory 16 or memories 11, 16 may also beconfigured to store data structures, configuration data, encryptiondata, historical system operations information, or any other specific orgeneric non-program information described herein.

Because such information and program instructions may be employed toimplement one or more systems or methods described herein, at least somenetwork device aspects may include nontransitory machine-readablestorage media, which, for example, may be configured or designed tostore program instructions, state information, and the like forperforming various operations described herein. Examples of suchnontransitory machine-readable storage media include, but are notlimited to, magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks; magneto-optical mediasuch as optical disks, and hardware devices that are speciallyconfigured to store and perform program instructions, such as read-onlymemory devices (ROM), flash memory (as is common in mobile devices andintegrated systems), solid state drives (SSD) and “hybrid SSD” storagedrives that may combine physical components of solid state and hard diskdrives in a single hardware device (as are becoming increasingly commonin the art with regard to personal computers), memristor memory, randomaccess memory (RAM), and the like. It should be appreciated that suchstorage means may be integral and non-removable (such as RAM hardwaremodules that may be soldered onto a motherboard or otherwise integratedinto an electronic device), or they may be removable such as swappableflash memory modules (such as “thumb drives” or other removable mediadesigned for rapidly exchanging physical storage devices),“hot-swappable” hard disk drives or solid state drives, removableoptical storage discs, or other such removable media, and that suchintegral and removable storage media may be utilized interchangeably.Examples of program instructions include both object code, such as maybe produced by a compiler, machine code, such as may be produced by anassembler or a linker, byte code, such as may be generated by forexample a JAVA™ compiler and may be executed using a Java virtualmachine or equivalent, or files containing higher level code that may beexecuted by the computer using an interpreter (for example, scriptswritten in Python, Perl, Ruby, Groovy, or any other scripting language).

In some aspects, systems may be implemented on a standalone computingsystem. Referring now to FIG. 7, there is shown a block diagramdepicting a typical exemplary architecture of one or more aspects orcomponents thereof on a standalone computing system. Computing device 20includes processors 21 that may run software that carry out one or morefunctions or applications of aspects, such as for example a clientapplication 24. Processors 21 may carry out computing instructions undercontrol of an operating system 22 such as, for example, a version ofMICROSOFT WINDOWS™ operating system, APPLE macOS™ or iOS™ operatingsystems, some variety of the Linux operating system, ANDROID™ operatingsystem, or the like. In many cases, one or more shared services 23 maybe operable in system 20, and may be useful for providing commonservices to client applications 24. Services 23 may for example beWINDOWS™ services, user-space common services in a Linux environment, orany other type of common service architecture used with operating system21. Input devices 28 may be of any type suitable for receiving userinput, including for example a keyboard, touchscreen, microphone (forexample, for voice input), mouse, touchpad, trackball, or anycombination thereof. Output devices 27 may be of any type suitable forproviding output to one or more users, whether remote or local to system20, and may include for example one or more screens for visual output,speakers, printers, or any combination thereof. Memory 25 may berandom-access memory having any structure and architecture known in theart, for use by processors 21, for example to run software. Storagedevices 26 may be any magnetic, optical, mechanical, memristor, orelectrical storage device for storage of data in digital form (such asthose described above, referring to FIG. 6). Examples of storage devices26 include flash memory, magnetic hard drive, CD-ROM, and/or the like.

In some aspects, systems may be implemented on a distributed computingnetwork, such as one having any number of clients and/or servers.Referring now to FIG. 8, there is shown a block diagram depicting anexemplary architecture 30 for implementing at least a portion of asystem according to one aspect on a distributed computing network.According to the aspect, any number of clients 33 may be provided. Eachclient 33 may run software for implementing client-side portions of asystem; clients may comprise a system 20 such as that illustrated inFIG. 7. In addition, any number of servers 32 may be provided forhandling requests received from one or more clients 33.

Clients 33 and servers 32 may communicate with one another via one ormore electronic networks 31, which may be in various aspects any of theInternet, a wide area network, a mobile telephony network (such as CDMAor GSM cellular networks), a wireless network (such as WiFi, WiMAX, LTE,and so forth), or a local area network (or indeed any network topologyknown in the art; the aspect does not prefer any one network topologyover any other). Networks 31 may be implemented using any known networkprotocols, including for example wired and/or wireless protocols.

In addition, in some aspects, servers 32 may call external services 37when needed to obtain additional information, or to refer to additionaldata concerning a particular call. Communications with external services37 may take place, for example, via one or more networks 31. In variousaspects, external services 37 may comprise web-enabled services orfunctionality related to or installed on the hardware device itself. Forexample, in one aspect where client applications 24 are implemented on asmartphone or other electronic device, client applications 24 may obtaininformation stored in a server system 32 in the cloud or on an externalservice 37 deployed on one or more of a particular enterprise's oruser's premises.

In some aspects, clients 33 or servers 32 (or both) may make use of oneor more specialized services or appliances that may be deployed locallyor remotely across one or more networks 31. For example, one or moredatabases 34 may be used or referred to by one or more aspects. Itshould be understood by one having ordinary skill in the art thatdatabases 34 may be arranged in a wide variety of architectures andusing a wide variety of data access and manipulation means. For example,in various aspects one or more databases 34 may comprise a relationaldatabase system using a structured query language (SQL), while othersmay comprise an alternative data storage technology such as thosereferred to in the art as “NoSQL” (for example, HADOOP CASSANDRA™,GOOGLE BIGTABLE™, and so forth). In some aspects, variant databasearchitectures such as column-oriented databases, in-memory databases,clustered databases, distributed databases, or even flat file datarepositories may be used according to the aspect. It will be appreciatedby one having ordinary skill in the art that any combination of known orfuture database technologies may be used as appropriate, unless aspecific database technology or a specific arrangement of components isspecified for a particular aspect described herein. Moreover, it shouldbe appreciated that the term “database” as used herein may refer to aphysical database machine, a cluster of machines acting as a singledatabase system, or a logical database within an overall databasemanagement system. Unless a specific meaning is specified for a givenuse of the term “database”, it should be construed to mean any of thesesenses of the word, all of which are understood as a plain meaning ofthe term “database” by those having ordinary skill in the art.

Similarly, some aspects may make use of one or more security systems 36and configuration systems 35. Security and configuration management arecommon information technology (IT) and web functions, and some amount ofeach are generally associated with any IT or web systems. It should beunderstood by one having ordinary skill in the art that anyconfiguration or security subsystems known in the art now or in thefuture may be used in conjunction with aspects without limitation,unless a specific security 36 or configuration system 35 or approach isspecifically required by the description of any specific aspect.

FIG. 9 shows an exemplary overview of a computer system 40 as may beused in any of the various locations throughout the system. It isexemplary of any computer that may execute code to process data. Variousmodifications and changes may be made to computer system 40 withoutdeparting from the broader scope of the system and method disclosedherein. Central processor unit (CPU) 41 is connected to bus 42, to whichbus is also connected memory 43, nonvolatile memory 44, display 47,input/output (I/O) unit 48, and network interface card (NIC) 53. I/Ounit 48 may, typically, be connected to keyboard 49, pointing device 50,hard disk 52, and real-time clock 51. NIC 53 connects to network 54,which may be the Internet or a local network, which local network may ormay not have connections to the Internet. Also shown as part of system40 is power supply unit 45 connected, in this example, to a mainalternating current (AC) supply 46. Not shown are batteries that couldbe present, and many other devices and modifications that are well knownbut are not applicable to the specific novel functions of the currentsystem and method disclosed herein. It should be appreciated that someor all components illustrated may be combined, such as in variousintegrated applications, for example Qualcomm or Samsungsystem-on-a-chip (SOC) devices, or whenever it may be appropriate tocombine multiple capabilities or functions into a single hardware device(for instance, in mobile devices such as smartphones, video gameconsoles, in-vehicle computer systems such as navigation or multimediasystems in automobiles, or other integrated hardware devices).

In various aspects, functionality for implementing systems or methods ofvarious aspects may be distributed among any number of client and/orserver components. For example, various software modules may beimplemented for performing various functions in connection with thesystem of any particular aspect, and such modules may be variouslyimplemented to run on server and/or client components.

The skilled person will be aware of a range of possible modifications ofthe various aspects described above. Accordingly, the present inventionis defined by the claims and their equivalents.

What is claimed is:
 1. A system for multitemporal data analysis,comprising: a computing device comprising a memory, a processor, and anon-volatile data storage device; a directed computational graph servicemodule comprising a first plurality of programming instructions storedin the memory and operable on the processor, wherein the first pluralityof programming instructions, when operating on the processor, causes thecomputing device to: convert input data, comprising a combination ofstored data and streaming data, into a directed computational graph,wherein: the directed computational graph comprises nodes representingdata transformations and edges representing messaging between the nodes;and the data transformation of each node is shared among a plurality ofinstantiated workers; and analyze the directed computational graph todetermine whether to batch process or real-time process portions of theinput data based on measuring shared state information among the nodes,edges, and instantiated workers of the directed computational graph; ageneral transformer service module comprising a memory, a processor, anda plurality of programming instructions stored in the memory thereof andoperable on the processor thereof, wherein the programmableinstructions, when operating on the processor, cause the processor to:receive batch processing data from the directed computational graphservice module; and perform batch processing of the batch processingdata according to a pre-determined first data processing workflow; and adecomposable transformer service module comprising a memory, aprocessor, and a plurality of programming instructions stored in thememory thereof and operable on the processor thereof, wherein theprogrammable instructions, when operating on the processor, cause theprocessor to: receive real-time processing data from the directedcomputational graph service module; and perform real-time processing ofthe real-time processing data according to a pre-determined second dataprocessing workflow.
 2. The system of claim 1, wherein a function isexecuted based at least in part by the results of data processing by thegeneral transformer service module and decomposable transformer servicemodule.
 3. The system of claim 1, wherein at least a portion of theinput data comes from a social media source.
 4. The system of claim 1,wherein at least a portion of the data input into the system comes fromactions of a user while using an application.
 5. The system of claim 1,wherein at least a portion of the input data from a news outlet.
 6. Thesystem of claim 1, wherein at least a portion of the input data comesfrom a distributed database.
 7. A method for multitemporal dataanalysis, comprising the steps of: converting, using a directedcomputation graph service module operating on a computing device, inputdata, comprising a combination of stored data and streaming data, into adirected computational graph, wherein: the directed computational graphcomprises nodes representing data transformations and edges representingmessaging between the nodes; and the data transformation of each node isshared among a plurality of instantiated workers; analyzing, using thedirected computational graph module, the directed computational graph todetermine whether to batch process or real-time process portions of theinput data based on measuring shared state information among the nodes,edges, and instantiated workers of the directed computational graph;receiving batch processing data from the directed computation graphservice module, using a general transformer service module operating onthe computing device; performing batch processing of the batchprocessing data according to a pre-determined first data processingworkflow, using the general transformer service module; receivingreal-time processing data from the directed computation graph servicemodule, using a decomposable transformer service module operating on thecomputing device; and performing real-time processing of the real-timedata according to a pre-determined second data processing workflow,using the decomposable transformer service module.
 8. The method ofclaim 7, wherein a function is executed based at least in part by theresults of data processing by the general transformer service module anddecomposable transformer service module.
 9. The method of claim 7,wherein at least a portion of the input data comes from a social mediasource.
 10. The method of claim 7, wherein at least a portion of theinput data comes from actions of a user while using an application. 11.The method of claim 7, wherein at least a portion of the input datacomes from a news outlet.
 12. The method of claim 7, wherein at least aportion of the input data comes from a distributed database.